Korean Part-of-speech Tagging Based on a Hidden Markov Model

In this paper, we describe a method for assigning a part-of-speech tag in Korean to each morpheme. The method is based on a hidden Markov model which can be trained without using any tagged corpus. To relax the amount of computation to process multiple observation sequences, which are extraordinarily occurred in Korean part-of-speech tagging, we develop a revised Viterbi algorithm for determining the most promising tag sequence. Experimental results show that the accuracy of the model is approximately 90% on average. The performance is no better than those of English tagging systems. This is due to the partially free word order feature of Korean and the lack of training data. However, to the best of our knowledge, this is the rst Korean POS tagging system which can be trained without using tagged corpus. Many words have ambiguous part-of-speech(POS) tags. For example, `time' in English can be a noun, a verb, or an adjective. In many cases, such ambiguities can be resolved using contextual information. For example, \Time is money.", the word`time' can be determined to a noun. The assignment of the correct POS tag to each word in a text is called part-of-speech tagging. The part-of-speech tagger is a system that selects the most appropriate POS tag for each word using the contextual information. POS tagging has been treated by several approaches; rule-based approaches, statistical approaches, neural network approaches and so on. We deal with the POS tagging problem by the statistical method which is described in terms of a Markov model. Hidden Markov modeling permits us to compute the most probable sequence of state transitions , which is the most likely sequence of POS tags for a given sentence. POS tagging in Korean has diierent aspects from that in English. In Korean, most word phrases 1 consist of more than one morpheme. In order to assign POS tags to the morphemes in each word phrase, we should morphologically analyze the word phrase in advance. One word phrase, however, may be analyzed in several diierent ways due to lexical ambiguities. Furthermore , each analyzed result may consist of the different number of morphemes. POS tagging in Korean can be done by the unit of a word phrasee12]. However , some problems in the word phrase-based tagging are the followings: Because all possible tags for word phrases cannot be predicted in advance, whenever a new tag for a word phrase is …